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MULTI-DIMENSIONAL PATTERN RECOGNITION BY USE OF DIGITAL 

HOLOGRAPHY 

TECHNICAL FIELD 

This invention relates multi-dimensional object, pattern or data recognition by 
use of digital holography. 

BACKGROUND OF THE INVENTION 

Optical correlation methods have proved to be very useful in the design of • 
two-dimensional (2D) pattern-recognition applications (A. VanderLugt, IEEE Trans. 
Inf. Theory IT-10, 139 (1964); J. L. Horner and P. D. Gianino, Appl. Opt. 23, 812 
(1984); D. Psaltis, E. G. Paek, and S. S. Venkatesh, Opt. Eng. 23, 698 (1984); Ph. 
Refrefgier, Opt. Lett. 15, 854 (1990) and J. W. Goodman, Introduction to Fourier 
Optics (McGraw-Hill, New York, 1996) which are incorporated herein by reference 
in their entirety). 

Recently, there has been increasing interest in three-dimensional (3D) optical 
information processing because of its vast potential applications. Several methods 
have been proposed to extend optical correlation methods to three-dimensional object 
recognition. One approach consists of processing different 2D projections of both a 
three-dimensional scene and a three-dimensional reference object by use of 
conventional 2D Fourier methods (A. Pu, R. Denkewalter, and D. Psaltis, Opt. Eng. 
36, 2737 (1997) which is incorporated herein by reference in its entirety). 

Other methods also involve the acquisition of different 2D perspectives, but 
the recognition is performed by three-dimensional Fourier-transform methods (J. 
Rosen, Opt. Lett. 22, 964 (1997) which is incorporated herein by reference in its 
entirety). This approach has analogies to incoherent-light three-dimensional imaging 
based on spatial coherent functions (H. Arimoto, K. Yoshimori, and K. Itoh, Opt. 
Commun. 170, 319 (1999) which is incorporated herein by reference in its entirety). 
Fourier-transform profilometry has also been applied to three-dimensional shape 
recognition (J. J. Esteve-Taboada, D. Mas, and J. Garcia, Appl. Opt. 22, 4760 (1999) 
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which is incorporated herein by reference in its entirety). 

However, holography, seems to be a more attractive method of performing 
three-dimensional image recognition, since a single hologram is able to record three- 
dimensional information on the object, avoiding sequential recording of several 2D 
5 perspectives and maintaining the phase (H. J. Caulfield, ed., Handbook of Optical 

Holography (Academic, London, 1979) which is incorporated herein by reference in 
its entirety). 

SUMMARY OF THE INVENTION 

£3 A method and system for recognizing multi-dimensional objects, patterns or 

J JO sets of data utilizing digital holography is disclosed. The method comprises 
generating a hologram of a reference object; generating a hologram of an input object; 
CO and correlating the hologram of the reference object with the hologram of the input 

?S object generating thereby a set of correlation peaks. The presence of a correlation 

peak indicates similarity between the reference object and the input object and the 
lis lack of the presence of a correlation peak indicates dissimilarity between the reference 
~;t object and the input object. The method may be used to determine a change in a 

p kinematic property of an object. 

A storage medium is disclosed that is encoded with a set of data created by 
generating a hologram of a reference object, generating a hologram of an input object, 
20 and correlating the hologram of the reference object with the hologram of the input 
object generating thereby a set of correlation peaks. 

The system for performing pattern recognition comprises a light source for 
generating an optical beam; means for dividing the optical beam into a reference 
beam and an object beam; means for introducing a phase shift between the reference 
25 beam and the object beam; an object positioned within the object beam; a beam 

combiner for combining the reference beam and the object beam; and a detector for 
detecting the combination of the reference beam and the object beam. 
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EXPLANATION OF THE DRAWINGS 



Figure 1 is a schematic representation of a phase-shifting interferometer for 
recording an inline digital hologram of a three dimensional object. 

Figure 2 is a schematic representation of the relationship between the 
displacement of a hologram window and the angle of view of the hologram. 

Figure 3 A is a first reconstruction of a three dimensional object from a digital 
hologram in a plane orthogonal to a detector. 

Figure 3B is a second reconstruction of a three dimensional object from a digital 
hologram in a plane orthogonal to a detector. 

Figure 4 A is the autocorrelation by digital holography of the reconstructed three 
dimensional object represented in Figure 3 A. 

Figure 4B is the cross correlation by digital holography of the three-dimensional 
objects represented in Figures 3 A and 3B. 

Figure 5 A is a conventional two-dimensional autocorrelation of the image of the 
two dimensional object in Figure 3 A. 

Figure 5B is a conventional two-dimensional cross correlation of the images of the 
two-dimensional objects in Figures 3A and 3B. 

Figure 6A is the correlation peak versus the angle of view used to measure three 
dimensional object orientation for the correlation of the three-dimensional object in 
Figure 3 A with a rotated version of the same three-dimensional object obtained by 
digital holography. 

Figure 6B is the correlation peak versus the angle of view for the correlation of 
the three-dimensional object of Figure 3 A with a rotated version of the same three- 
dimensional object giving the maximum correlation peak. 

Figure 7 is a schematic representation of a computer network connected to the 
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phase-shifting holographic system of Figure 1. 

Figure 8 is a representation of the four step method of inducing a phase shift in the 
reference beam of the interferometric system of Figure 1. 

Figure 9A is a first schematic representation of an optical correlation of a 
5 hologram of a reference object and a hologram of an input object. 

Figure 9B is a second schematic representation of an optical correlation of a 
hologram of a reference object and a hologram of an input object. 

r3 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

An optical system for the digital holographic recording of a three-dimensional 
FBO object, a pattern or a set of data is based on an interferometer, such as a Mach- 
f 3 Zehnder interferometer 100, and is depicted in Figure 1. Polarized light 104 

^ emanating from a laser 102, such as an Argon laser, a helium-neon laser an infrared 

C3 laser or other such laser, is divided into an object beam 104b and a reference beam 

I j 104a. The object beam 104b illuminates a still or moving three-dimensional reference 

5 15 object 130, while the reference beam 104a forms an in-line interference pattern 
U together with the light 104d diffracted by the reference object 130. The reference 

object 130 may comprise, for example, an optical image, a digitized image, a 
computer generated image, a one-dimensional or multi-dimensional object, pattern or 
set of data, an electrical signal or an optical signal. The object 130 may also include 
20 two or three-dimensional phase objects or color objects and is shown as a three- 
dimensional object by way of exemplification. 

The interference pattern, or hologram, is formed or recorded on a detector or 
storage medium 128, such as a CCD camera. In Figure 1, the reference numeral 108 
denotes a spatial filter, 110 denotes a lens, 112 is a diaphragm, 114 and 126 are beam 
25 splitters, 106, 120, 122, 124, are plane mirrors, 116 and 118 are phase retarders with 
retardation of A/2 and A/4 respectively, where A is the wavelength of the light 
emanating from the laser 102. The reference object 130 is located at a distance d from 
a recording plane 210 in the CCD camera 128. The CCD camera 128 is in turn in 
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signal communication with a monitor 136 and a computer 134. 

Consider the opaque three-dimensional reference object 130, as a complex 
amplitude distribution Uo(x f y, z), where x f y are the transverse coordinates and z 
denotes the paraxial distance from the reference object 130 to the output plane 210 of 
the CCD camera 128. In the Fresnel approximation, neglecting secondary diffraction, 
the complex amplitude distribution, H 0 (x t y,), due to the object beam 104d and 
recorded in the output plane 210 can be evaluated from the following superposition 
integral: 

H Q (x,y) = A H (x,y)exp[i<p H (x,y)] 

= ^lfp 0 (x\y;z)-exp(i^z) (1) 

Z A 



x exp 



dbCdydz 



W The complex field generated by the reference beam 104a at the output plane 

£3 210 can be written as R(x, y; On) = A R exp[*0 + c^)], where^ is the constant phase 

1 1 when both fast axes of the phase retarders 116, 1 1 8 are aligned with the direction of 

polarization and is the phase shift in other configurations of the phase retarders 
15 1 16, 1 1 8. As seen in Figure 8, by adjusting the relative position of the slow and fast 
axes of the phase retarders 116, 118, phase shifts of = 0, %/2, 7t, and 3tt/2 are 
produced without altering the constant amplitude, A Ri of the reference beam 104a. 
The phase retarders 116, 118 allow for the modulation or shifting of the phase, of 
the reference beam 104a with respect to the object beam 104b. In this manner, phase 
20 shifts, are introduced into the reference beam 104a with respect to the object beam 
104b by the relative positioning of the fast and slow axes of the retarder plates 116, 
118 with the polarization of the incident light 104a. The aforesaid phase shifts, 
may also be introduced into the reference beam 104a with respect to the object beam 
104b by other methods or apparatus, such as the use of a liquid crystal chip in place of 
25 the phase retarders 116, 1 1 8 or by the movement of mirror 1 24. 
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The complex amplitude given by Eq. (1) is measured by recording four 
interferometric patterns, or holograms, I n (x t y; aj = \Ho(x, y) + R(x t y; | 2 . 

The holograms, I n (x, y; o^) s formed by the combination of H 0 (x t y) and R(x, y; 
are recorded digitally, or on film, as a hologram at the output plane 210. Such 
recording may be in the form, for example, of tangible storage media, such as optical 
storage materials (e.g., photorefractive materials), or digitally on floppy diskettes, 
CD-ROMs, hard drives, optically or electrically addressable spatial light modulators, 
charge coupled devices or any other computer-readable storage medium or processing 
unit addressable across a distributed environment such as a computer network system. 
It will be recognized that the holograms formed at the output plane 210 are not limited 
to the in-line holograms of Figure 1, but may also be formed by off-axis holograms as 
is well known in the art. 

The phase difference between the object beam 104b and the reference beam 
104a is given by 



^(x,jv)-^ = arctan 



I(x,y;-37r/2)-I(x,y-7r/2) 
I(x,y\0)-I(x,y-7u) 



(2) 



The amplitude of the interference pattern of the object beam 104d and the 
reference beam 104a can be calculated from: 



A H (x,y)A R = 



1 l(x 9 y\0)-Hx,y\-K) 
4 cos[<f> H (x, y)-<p)] 



(3) 



where the cosine function is obtained from Eq. (2). The parameters <p and A R are 
constant factors which can be replaced with 0 and 1, respectively. 

Different regions or windows within the digital hologram of the reference 
object 130, record different perspectives, views or segments of the reference object 
130. These different perspectives, views or segments of the reference object can be 
reconstructed by the numerical propagation of Ho(x t y). The discrete complex 



UCT-0016 



amplitude distribution of the reconstructed reference object, Ud(m t n; a Xj a y ), at any 
plane in the object beam 104d, within the paraxial approximation and within the angle 
limitations imposed by the size of the CCD 128, can be numerically computed, aside 
from constant factors, by the discrete Fresnel transformation: 



U 0 i (m,n;a xJ a y ) = cxp 



in 



Xd 



-(Ax 2 m 2 +Ay 2 n 2 ) 



N-l N-l 
m '=On'=Q 



ITU 



; (Ax t2 m' 2 +Ay ,2 rt' 2 ) 



exp 



— iln 



mm- nn 



(4) 



i where d' = -d\ (m\ ri) are discrete spatial coordinates in the plane of the hologram 

210; (m f n) correspond to discrete spatial coordinates in the plane of the reconstructed 

jlO object and N x and N y are the number of samples in the x and y directions in the plane 
of the reconstructed object, respectively. The spatial resolution in the plane of the 

3 hologram 210 is (Ax', by'\ and the spatial resolution in the plane of the reconstructed 

object is (Ax, Ay). H 0 '(m, n; a x , a y ) denotes the amplitude distribution of the object 
over a window 212 defined within the hologram 210 and used for the reconstruction 
15 of the reference object 130 (Figure 2). Hd(m, n; a X) a y ) is modified by a linear phase 
factor, cxp\i27r{a x m + a y n)\ to take into account the direction of observation. 

Thus, 



H 0 '(m 9 n;a x ,a ) = H Q (m, n)rect 



r \ 
m-a„ n- a v 



'y J 



x cxp[i27r{a x m + a y rif[ 



(5) 



20 

wherein rect(g,h) is the rectangle function. In Eqs. (4) and (5), (a x , a y ) are the pixel 
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coordinates of the window 212 and (b x , b y ) denote the transverse size of the window 
212. Figure 2 shows the relationship between the displacement of the hologram 
window 212 and the angle of view along the direction of observation 204. Angles of 7 
and /? are equal to a,Ax7 d and a^AyV d, respectively. The minimum angle, 
between different perspectives of the reference object 130 depends on the CCD pixel 
size, Ax', and the distance d, such that %„•„ = Ax 7 d. A similar relationship applies for 
the y direction for the angle ft i.e., (3 min = Ax 7 d. The maximum angle, y maXt is a 
function of the CCD pixel size and the window size, such that y maXt = (N x - b x ) Ax7 d\ 
and similarly for the angle jS in the y direction, i.e., t3 max , = (N y - b y ) Ay 7 d. It will be 
appreciated that the resolution of the reconstructed images, Ax and Ay are also 
affected by the window size, i.e., Ax = \d/(b x Ax') and Ay = \dl(b y ky'). Equation (4) 
can be numerically evaluated with a fast Fourier-transform method. An alternative 
method of evaluating Eq. (4) is to use the propagation transfer function method: 



U 0 Xm,n;a x ,a y ) = F- l (F[H 0 (m,n;a x ,a y )]xexp 



u v 

■ + - 



(AxN x ) (AyN ) 



(6) 



where F denotes the fast Fourier transformation and u and v are discrete spatial 
frequencies. 

To perform recognition of, or discrimination between, three-dimensional 
objects, patterns or sets of data, a Fourier-matched filter approach is applied to the 
information obtained by digital holography. Consider a three-dimensional input 
object 132 whose complex amplitude distribution is given by U p (x, y t z). The input 
object 132 may comprise an optical image, a digitized image, a one dimensional set of 
data, a two dimensional set of data, a multi-dimensional set of data, an electrical 
signal or an optical signal and is shown as a three-dimensional object by way of 
exemplification. The input object 132 is also located at a distance d from the CCD 
128, and has a real term in a corresponding Fresnel hologram 210, ofH p (x, y). 
Correlation between different views of the input object 132 and a given view of the 
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reference object 130, i.e., object recognition, can be performed by defining a window, 
Hp (x, y; a*', a/), within the Fresnel hologram, H p (x, y\ of the input object 132 by 
the use of Eq. (5), followed by reconstructing the input object 132 with Eq. (4) 
(yielding the reconstructed input object Up(x f yz)) and computing the correlation 
between U p '(x, y z) and Ud(x, y z). Alternatively, by use of Eq. (6), we can write* the 
correlation intensity of the reference amplitude, Ud (x, y; a,; a y \ with that of the 
input, Up (x, y; a,', a y '\ generated from the holograms: 

C„(x 9 y 9 a x9 a/ 9 a x ^ 

= \F- X {F[H 0 \x,y;a x ,a y )]x F * [H p '(x 9 y;a x \a;)f 

(7) 

Thus, by performing the correlation between different regions or windows defined 
within the hologram of the reference object 130 and the hologram of the input obje.ct 
132, properly modified by a linear phase factor, an evaluation of the correlation 
between different perspectives, views or segments of the reference object 130 and 
input object 132 is performed. It will be appreciated that rough objects involve fast 
fluctuations of the reconstructed phase during translations, thus reducing the shift 
invariance. Nevertheless, high sensitivity is obtained for small object motion. This is 
illustrated by measuring a small rotation of a rough three-dimensional object. 

A three-dimensional object recognition experiment was performed with two 
reproductions of cars with an approximate size of 25 mm * 45 mm. The objects were 
located at an approximate distance, d, of about 865 mm from the output plane 210. 
The pictures in Figures 3 A and 3B were obtained by recording a digital hologram of 
each car, denoted H 0 {x, y) and H p (x, y). Reconstruction was performed by computing 
Eq. (4) with a fast Fourier-transform method applied to both of the holograms. 

Figure 4A shows a correlation peak 230 in a plot 232a of the autocorrelation of 
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the reconstructed object shown in Figure 3 A, which was performed by use of a digital 
hologram, Hd (x t y; 0, 0), obtained by placement of a window with size b x = b y = 256 
pixels centered at the origin of H 0 (x, y) (see Eq. (5)). Figure 4B shows a plot 232b of 
the cross correlation of the three-dimensional object in Figure 3A with that in Figure 
5 3B 3 which was also obtained by digital holography. No correlation peak is seen in 
Figure 4B. The reference hologram is the previous digital hologram Hd (x, y; 0, 0), 
and the input, Hp (x, y; 0, 0), is obtained by placement of a window with the same 
size centered at the origin of H p {x, y). Both plots are normalized to the same value. 
By way of comparison, Figure 5 A shows the conventional two-dimensional 
_10 autocorrelation 232c and Figure 5B the conventional two-dimensional cross 
%3 correlation 232d, using the objects in Figures 3A and 3B registered as two- 

=ij dimensional intensity images. 

SO In Figure 9 A the correlation of the hologram 222 of the reference object 130 

ifi with the hologram 224 of the input object 132 may be performed optically rather than 

115 digitally. An electrically addressable spatial light modulator 220a is interfaced with 
W the detector 128. The digital holograms 222, 224 of the reference and input objects 

£3 are displayed on the electrically addressable spatial light modulator 220a and the 

H correlation of the holograms is constructed in a correlation plane by a reference wave 

104g. In Figure 9B, the holograms 222, 224 are formed by an optically addressable 
20 spatial light modulator 220b such as a liquid crystal light valve or a ferro-electric 

spatial light modulator. The hologram 224 of the input object 132 and the hologram 
222 of the reference object 130 are read out from the output of the optically 
addressable spatial light modulator 220b by the reference wave 104g and the 
correlation of the holograms 222, 224 is formed at the correlation plane 232. The 
25 following are incorporated herein by reference: B. Javidi and J. L. Horner, "Real-time 
Optical Information Processing" Academic Press, 1994; U.S. Patent No. 4,832,447, 
entitled Joint Transform Image Correlation Using a Nonlinear Spatial Light 
Modulator at the Fourier Plane; U.S. Patent No. 4. 5,841,907, entitled Spatial 
Integrating Optical Correlator for Verifying the Authenticity of a Person, Product or 
30 Thing; U.S. Patent No. 5,699,449, entitled Method and Apparatus for Implementation 
of Neural Networks for Face Recognition; U.S. Patent No. 5,367,579, entitled Method 
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of Removing Spurious Responses from Optical Joint Transform Correlators; U.S. . 
Patent No. 5,1 19,443, entitled Nonlinear Joint Transform Optical Correlator Having 
Varying Degrees of Nonlinearity; U.S. Patent No. 5,485,312, entitled Optical Pattern 
Recognition System and Method for Verifying the Authenticity of a Person, Product 
5 or Thing; B. Javidi, "Optical Information Processing" in Encyclopedia of Electrical 
and Electronic Engineering, Volume on Optics, John Wiley, 1999 and B. Javidi, 
"Optical Neural Networks " in Encyclopedia of Electrical and Electronic Engineering, 
Volume on Neural Networks, John Wiley, 1998 and B. Javidi and E. Tajahuerce, 
"Three Dimensional Image Processing And Reconstruction" International Symposium 
10 On Photonics For Aerospace Application Of Optics, SPIE Proceedings Vol. 4043, 
J5 Orlando Fla., April 24 - 28, 2000 

if. It will be appreciated that the methods disclosed herein can be used to measure 

CS changes in the kinematic properties (translation or rotation) of the three-dimensional 

in input object 132 with respect to the reference object 130 by adjusting the parameters 

115 in the phase factor applied to the window, H p ' {x, y; a*', a y '\ defined within the 
l-J Fresnel hologram, H p (x, y\ of the input object 132. In particular, Figure 6 shows the 

£3 correlation obtained by digital holography between the three-dimensional object in 

s ;i Figure 3 A, which was used as a reference, and the same slightly rotated three- 

dimensional object used as input. A window with a size, for example of, b x — b y = 256 
20 pixels and centered at the origin of the hologram of the reference object was utilized. 

The same window is shifted to different positions in the hologram of the input object. 
Figure 6 A shows the maximum value of the correlation 232e as a function of the 
displacement (a x , a y ) of the window in the hologram of the input. Apeak 232e is 
obtained for a x = -12, a y = -2, corresponding to an angle of view (7, (3) = (0.007°, 
25 0.001°). Figure 6B shows the correlation 232f for this perspective of the three- 
dimensional object. The sharp peak 232f indicates that the object is only rotated with 
respect to the axial view of the reference. 

Thus a method of determining a change in a kinematic property of an object, is 
realized by generating a first hologram of the object, generating a second hologram of 
30 the object, correlating the first hologram with the second hologram, generating 
thereby a set of correlation peaks and analyzing the set correlation peaks. The 
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aforesaid method may be accomplished by defining a window within the first 
hologram, defining a window within the second hologram and correlating the first and 
second windows, generating thereby a set of correlation peaks and analyzing the set 
correlation peaks. The aforesaid method may further comprise autocorrelating the first 
5 hologram and comparing the autocorrelation of the first hologram with the correlation 
of the first hologram with the second hologram. 

Referring to Figure 7, the detector 128 is shown connected to a computer or 
communications network 400, such as a local area network (LAN) or a wide area 
network (WAN) via a computer 134. The detector 128 may also be connected directly 
10 to a liquid crystal display (LCD), liquid crystal television (LCTV) or an electrically or 
5 optically addressable spatial light modulator or ferroelectric spatial light modulator 

y 412a. The computer network 400 includes a plurality of similar client personal 

Sf computers 404 connected to a server 410 from remote geographical locations by 

H wired or wireless connections, by radio based communications, by telephony based 

communications, or by other network-based communications. The computer 134 may 
1 also be connected directly to another like computer 134 or to a display device 412, 

3 such as a liquid crystal display (LCD), liquid crystal television (LCTV) or an 

7 electrically or optically addressable spatial light modulator (SLM) for 3D optical 

reconstruction of the objects 130, 132 or patterns or sets of data. The computer 
20 network 400 is in turn similarly connected to other computers 502 or networks 504 
through the Internet 500. The computers 134, 404, 502, display devices 136, 412, 
412a and other devices of the networks 400, 500, 504 are configured to execute 
program software, that allows them to send, receive, store, record and process the 
holograms, H 0 {x, y) and H p {x f y), either separately or in combination, between and 
25 amongst themselves via the networks 400, 504 or the Internet 500. Such processing 
includes, for example, image compression and decompression, filtering, contrast 
enhancement, image sharpening, noise removal and correlation for image 
classification. Decompressed images may be displayed on display devices such as 
liquid crystal displays, liquid crystal TVs or electrically or optically addressable 
30 spatial light modulators. 

Based upon the foregoing description, an optoelectronic holographic method 
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and system for performing pattern recognition of three-dimensional objects, or sets of 
data, by use of digital phase-shifting holography has been presented. This method is 
based on the ability of a digital plane hologram to reconstruct different perspectives of 
a three-dimensional object. The method allows one to obtain three-dimensional 
information on reference and the input objects. Three-dimensional pattern recognition 
is carried out by use of a digital matched filter technique applied to the holographic 
information. The holographic data can be transmitted through conventional digital 
communications channels to remote locations and reconstructed therefrom digitally or 
optically. 

Utilizing phase-shifting interferometry, (J. H. Bruning, D. R. Herriott, J. E. 
Gallagher, D. P. Rosenfeld, A. D. White, and D. J. Brangaccio, Appl. Opt. 13, 2693 
(1974) and I. Yamaguchi and T Zhang, Opt. Lett. 22, 1268 (1997) which are 
incorporated herein by reference in their entirety) the methods herein electronically 
record the complex amplitude distribution generated by a three-dimensional object at 
a single plane located in the Fresnel diffraction region. Similarly, a digital hologram 
of a three-dimensional reference pattern is recorded to be used as a correlation filter. 
Thus, pattern recognition by use of three-dimensional information can be performed 
by application of correlation methods to digital holograms. 

While preferred embodiments have been shown and described, various 
modifications and substitutions may be made thereto without departing from the spirit 
and scope of the invention. Accordingly, it is to be understood that the present 
invention has been described by way of illustration only, and such illustrations and 
embodiments as have been disclosed herein by reference are not to be construed as 
limiting the claims. 
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